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Abstract 


Background: Adaptive and personalized learning technologies are on the rise in health education. 
However, to reach the potential of these technological innovations, novel learning designs are necessary, 
which take new possibilities and constraints into account. 


Aim: In this scoping review we answer the question: What characterizes learning designs where adaptive 
learning technologies have been blended into nursing education? 

Methods: Using the terms adaptive learning and nursing education with synonyms in combination, a 
comprehensive search in five databases were conducted. Initial search identified 340 records. 22 articles 
were identified as relevant and screened in full text reading and included. Final number of papers included 
in the review was six. 

Conclusion: We conclude that duration, engagement, placement of interaction in time and agency are the 


most commonly addressed parts of the learning design. We also find that there is a lack of pedagogical 
justification of the learning designs used. 


Keywords: computer-assisted instruction, nursing education, programmed instruction, self-directed 
learning, adaptive learning 


INTRODUCTION 


With rapid advances in machine learning, artificial intelligence and big data, the potentials of educational 
technology changes in equal measure. A particular example of this is the new trend of ‘adaptive learning 
technologies’ (ALT), and ‘technology-enhanced personalized learning’. Such educational technologies aim to 
use input data from students to present content to be learned in a way that best suits the learners’ specific 
needs and conditions. 


In the context of health professions’ education, ALT have also gained ground, particularly in medical 
education and in further education. A recent meta-analysis hereof concludes that the adaptive learning 
technologies seem to have a positive impact on the desired learning outcomes (Fontaine et al., 2019). Thus, 
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there is reason to believe that adaptive learning technologies will become a more widespread kind of learning 
resource in nursing education in the future. 


However, research within the field of education technology has also repeatedly shown that the impact of 
technology cannot be separated from the teaching strategies and learning design, with which it is used 
(Kirkwood & Price, 2013). With adaptive learning technology, the technology holds the potential to substitute 
and augment the actions of the human instructor in some regards: it asks questions, the students respond 
and the technology provides personalized explanations, if the students are unable to answer correctly. This 
raises a question about the best way to integrate ALT into nursing education with learning of both clinical, 
procedural skills and declarative knowledge? Should it be considered just a replacement of the textbook? Is 
it a new kind of continuous testing system? How do the instructors make the most of the data and content 
provided by the technology? How do instructors make sure that adaptive learning technology relates to and 
ensures a transfer from declarative knowledge to clinical practice? In this review, we provide an overview of 
key features and factors to address in learning design of nurse education with ALT. We draw on the concept 
of learning design, which explicitly addresses how learning and education is arranged with the use of 
technology, and draws particularly attention to the sequential structure and supposed actions of instructors 
and students. A learning design is defined as: “[...] a representation of what happens in a teaching and 
learning session to help learners achieve specified learning outcomes. It is often structured as a sequence of 
learning activities that can be shared with others. For online and blended learning, this approach is able to 
show, not only what is happening when the teacher is with the learners, but also what learners should be 
doing when the teacher is absent and they are being supported by technology”(Laurillard et al., 2018, p. 
1046). 


In this review we identify papers that describe the learning design of their interventions in nursing education 
with ALT. The scope of the review is to identify and discuss how the technology pedagogically is blended into 
the learning design to support and inform practice by addressing the research question: What characterizes 
learning designs where ALT have been blended into nursing education? 


THE REVIEW 
Aim 


The aim of this review is to examine the learning design through which ALT have been implemented in nursing 
education. 


Design 


The review has been conducted using the method of scoping reviews. Scoping reviews are distinguished from 
systematic reviews by having a broader interest of inquiry and being more inclusive in terms of research 
designs than a systematic review (Arksey & O’Malley, 2005). This is particularly relevant when researching 
an emerging field of technology, as studies may draw on different methodological approaches, have different 
interests of inquiry and yet still reveal findings about how to blend ALT into nursing education. The method 
of scoping reviews is particularly relevant when it comes to identifying potential gaps and point to directions 
that future research and practice can benefit from (Arksey & O’Malley, 2005). 


Search Methods 


Databases were searched in January 2021 to identify papers that explored ALT in nursing education. The 
following databases were searched (hits): CINAHL (39), EBSCO (1), Google Scholar (57), PubMed (11), Scopus 
(232). The search was conducted in collaboration with a librarian and in accordance with the syntax of each 
database with a string consisting of: 


[“adaptive learning” OR “personalized learning” OR “individualized learning” OR “intelligent tutoring system” 
OR “computer-assisted instruction”] AND [technology OR platform OR system OR technologies OR software] 
AND [“nurs* education” OR “nursing education” ]. 
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Thesaurus and indexed keywords were used where appropriate in each database. Furthermore, hand 
searching of reference lists in included studies was also conducted, returning additional 2 hits. 


Search Limits 


The search was limited to English language and peer-reviewed papers published from 2010 to 2021. 


Inclusion Criteria 


The papers were included if they met the following three criteria: 1) related to the context of nursing 
education 2) being about technology that algorithmically adapts to one or more of each student’s personal 
traits such as, e.g., proficiency or learning preferences 3) describe principles of learning design or an 
educational intervention. 


Exclusion Criteria 


Papers were excluded if the technology described was not adaptive (e.g. clickers or social media), or the 
adaption was not related to the students learning process (e.g. adaptive rehabilitation technologies for 
patient care), or the context was not nursing education (physicians, medicine or continuing education for 
graduated nurses). Particularly, most of the excluded articles were so, because they did not address the 
context of nursing education. 


Search Outcomes 


Initial search resulted in 340 papers, and 2 additional articles were found through handheld searching. After 
removal of duplicates, 327 papers remained for screening. Screening of title and abstract was undertaken by 
first author (BLA) based on the inclusion criteria. After this, 22 full text articles remained for assessment of 
eligibility. This process was undertaken by first author (BLA) and second author (RLJ) based on in- and 
exclusion criteria (exclusion with reasons). The process left 6 articles for qualitative synthesis in the scoping 
review. 
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Records identified through Additional records identified 
database searching through other sources 
(n= 340) (n= 2) 


Identification 


Records after duplicates removed 
(n= 327) 


Screening 


Records screened Records excluded 
(n= 327) (n= 305) 


Full-text articles assessed Full-text articles excluded, 
for eligibility with reasons 
(n= 22) (n= 16) 


Eligibility 


Studies included in 


qualitative synthesis 
(n= 6) 


Figure 1. Review process based on Moher et al. (2010) 


DATA CHARTING 


In a scoping review, a major concern is to identify and define the variables and themes by which each study 
is best described in order to address the research question (Arksey & O’Malley, 2005). By reading the six 
papers the following six themes have been identified as relevant: 


Motivation for use. What are the hypotheses of impact or desired outcomes of using adaptive technology 
in nursing? 


Educational context of intervention: What are the institutional and educational contexts in which ALT has 
been blended in? 


Research methods and data. What methods have been used in the studies and what is the sample size? 


Technological intervention: What characteristics and features of the adaptive technology are highlighted? 
Learning design intervention: What is the duration and extent of the blend, and what are the explicit 
and/or implied actions of students and instructors in relation to the use of adaptive technology and how 
has the course sequentially been built? 


Findings and suggestions: What are the identified potentials of using ALT in nursing education, and what 
might be addressed in the future? 


First, we present a chart with relevant findings from each included articles in a chart and afterwards we 
discuss relevant findings across the articles. 


4/11 


Andersen et al. / Contemporary Educational Technology, 2022, 14(1), ep333 


Table 1. Data charting 


nee Educational Research : is Ge f 
Motivation for Technological ; eS : Findings and suggestions 
Source context of methods and . : Learning design intervention i ; , 
use : . A intervention regarding learning design 
intervention sample size 
Hinkle et To compare Nursing program Randomized Algorithmic use of real 7-weeks of duration online | Students believe adaptive 
al. (2020) results with in control group time data from course. learning to improve their 
traditional pathophysiology experiment. student-platform Questions in AL-platform learning and would prefer 
methods of in the USA. Intervention interaction to were delivered before, during adaptive learning in future 
delivery and 2018. group n=35. determine the optimal and after a learning module. courses. 
assess student path through course Each AL-module could be 
perceptions of content for each repeated as many times as a 
adaptive student (Platform: learner’ chose@inorder‘to 
learning Realizelt) improve knowledge, with no 
specific engagement 
requirements. 
Hinkle et Enhance case- Nursing program Experimental Bayesianestimation Five sections consisting of In terms of engagement, 
al. (2018) based learning, in study in spring techniques that give two online, one blended and_ students tend to engage with a 
so that the pathophysiology (n=95), each student a one face-to-face. case-question once or twice 
complexity ofa inUSAinspring summer personal path through Case-specific questions were during a course, spending 
case is adjusted 2015. (n=22) and fall course content created developed so that they between 5 and 7 minute in each 
to each Gaurcetcontent (n=124) in by faculty. Questions simulated lab and diagnostic attempt. Balancing meaningful 
student's diesen can contain variables, Peer eithatetidenterodld case-questions aa are not too 
proficiency. specifically such as age, and thus encounter in practice. taxing is of great importance to 
pereeererce cases presented can be learning design. 
: aCe differentiated for Instructor encouraged 
Uefelles pi Met oric students (Platform: students to engage with a 
difficulty. Realizelt) i number of “nodes”, 
comprising an overall topic 
objective, but engagement 
was not counted as a part of 
students’ course grade. 
Course designed so that 
students worked individually. 
Instructors supported by 
expert instructional designers 
in creating content for the 
platform. 
Morente Todetermine Course in Randomized An adaptive self- 4-hours single session Intervention proved to be 
et al. the effectiveness pressure ulcer control group learning e-learning tool module, where students were effective, but no findings of 
(2014) of adaptive training in Spain. study with developed by the physically located inaspace relevance to learning design are 
learning Chosen because pre-test and research team, which with a computer each with no reported. 
compared to it is considered a post-test. N= adapts to the students’ intervention from the 
traditional on- challenging topic 73 (control skills and provides instructor. 
campus teaching for most group= 43, online tutorials 
methods. students. intervention (Platform: ePULab). 
group = 30). 
Simon- Prepare Second semester Retrospective Online, computer- Students had access tothe Data about students’ curricular 
Campbell students for adult health descriptive based platform with a system and could choose weakness can point instructors’ 
and NCLEX license —_ course. and large database of test between an older and newer attention to areas that need 
Phelan examand correlational questions calibrated in version of the technology. focus and remediation in 
(2016) optimize design. N = level of difficulty and in Attained mastery level was an teaching. For this to happen, 
prediction of 100. various formats important feature to guide data needs to come timely and 
students with (multiple choice, fill the and engage students. No there has to be a clear pathway 
likelihood of blank etc.). Adapts to further description of the to remediation. 
failing NCLEX. each student’s euEeDt learning design intervention. TRetrictors canicreaterc|aee 
level of understanding, assignments using collections of 
and after finishing the questions chosen for the 
quiz, the student sees REFER 
explanations of key 
concepts, answer keys Students can independently 
and results. (Platform: take quizzes and train their test- 
PrepU). taking skills. 


Policy stating af standardized 
use of the technology may be 
beneficial. 


Create regular, required 
assignments or associate a 
participation grade with 
quizzing activity. 
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Table 1 (continued). Data charting 


ahs Educational Research i se F 
Motivation for Technological ; hoes : Findings and suggestions 
Source context of methods and . . Learning design intervention 7 : , 
use : , ‘ intervention regarding learning design 
intervention sample size 
Malkemes To increase the Senior-level Retrospective According to each Lasted two semesters. Important to distinguish 
and number of first course ata descriptive students response, the Describes two ways, in which between whether technology is 
Phelan time passes in nursing school in and technology adjusts and the adaptive technology was used in formative or summative 
(2017) NCLEX-RN exam, USA correlational personalizes the used. Firstly, instructors can ways. Formative ways point 
make students design. N = 55. learning experience in integrate the technology in instructors’ attention to topics 
more self- terms of difficulty of | their teaching by creating or questions that need 
confident and to test items. Technology custom quizzes from the remediation, how student 
monitor calculates a mastery database, or set a particular engagement tends to increase 
students’ level for each student mastery level in a certain when adaptive learning is used 
preparation for (Platform: Wolters topic for students to achieve. in summative ways, where their 
the license Kluver PassPoint). Secondly, students can use it mastery level is considered part 
exam. for their own independent of the final course grade. 
study and create their own 
custom quizzes. 
Students were required to 
answer at least 25 questions 
per week first semester, and 
50 questions second 
semester. If they did not 
achieve at mastery level of 4, 
they were required to dobule 
the amount of questions 
answered. 
Their performance in the 
system was considered a part 
of the final grade. 
Presti and To reduce test Senior-level, Quasi- Technology establishes Course combined 112 hours Use of customized adaptive 
Sanko anxiety, and final adult health experimental and continually of in preceptorship in clinic | quizzes and remediation of 
(2019) strengthen the nursing course inretrospective updates a score of with didatic content content individualized for each 
students’ USA. pre-test post- students’ abilityand | emphasising question and student impacts learning 
knowledge base. test study.1 give them more answer sessions related to outcomes positively. 


control group, challenging questions _ their clinical experience 

3 intervention as they progress until a provided through the 

groups. preset target mastery adaptive technology. No 
level is further description reported. 
reached.Platform not 
mentioned. 


FINDINGS 


Motivation for Use 


There are various reasons for adopting ALT in nursing education. However, two motives are most prominent 
Firstly, a desire to determine whether ALT improves the learning outcomes compared to traditional teaching 
methods or not (Hinkle et al., 2020; Morente et al., 2014). Secondly to enhance the likelihood that students 
pass final license exams (Malkemes & Phelan, 2017; Presti & Sanko, 2019; Simon-Campbell & Phelan, 2016). 
A single study is pedagogically motivated by a hypothesis that adaptive technologies may provide an 
enhancement of case-based learning by personalizing cases to the students’ abilities (Hinkle & Moskal, 2018). 
Reading across the articles, it is evident that the motivation is driven by a summative rather than formative 
focus in the learning design, i.e., predominantly related to the students’ final achievement and to a lesser 
degree by an enhancement of their on-going learning process. 


Educational Context of Intervention 


The questions of geographical origin, subjects and stage are important aspects because the context of 
intervention determines the degree to which the studies are comparable to other cases and future, and 
learning design interventions may vary across cultures, subjects and stages within nursing education. Thus, 
the findings of the articles are embedded in particular cultures of nursing education, set at particular stages 
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within the education and tested in disparate subjects. Of the studies that report their geographical origin, 
five of them are based in nursing education in the USA, and a single in Spain. The subjects in which the ALT 
has been used are various, and only two state reasons why the specific courses have been chosen. Those two 
state that the subjects have been chosen for the intervention, because students usually find those subjects 
particularly difficult (Hinkle & Moskal, 2018; Morente et al., 2014). Only two studies report at which stage 
during education, ALT has been used. The two concerned have placed their interventions in the second 
semester (Simon-Campbell & Phelan, 2016) and in a final course (Presti & Sanko, 2019) respectively. 


Research Methods 


All of the studies are quantitative with variations in sample size and design, and aimed at calculating the 
effect of the adaptive technology intervention. Some have control groups and some do not, some have post- 
test and pre-test and some do not. The experimental nature of all the studies seems to cause a dominance 
of research design over learning design, and thus the way ALT have been put to use is governed by having a 
controlled set up rather than arguing and discussing what patterns of use may or may not be beneficial from 
a pedagogical perspective. 


Technological Intervention 


Four of the studies use commercially available software, one uses software developed by the research team 
(Morente et al., 2014) and one does not report the particular technology used (Presti & Sanko, 2019). The 
characteristics of adaptive technology highlighted by each study vary in the level of details by which the 
adaptive system is described. However, the dominant feature of the adaptive technologies is that they 
algorithmically determine the proficiency of each student, and based on this generate an individual learning 
path through the course content. As students improve, they are given harder questions. Some of the 
technologies used allow students to design quizzes themselves and provide the students with an overall 
mastery score (Malkemes & Phelan, 2017; Presti & Sanko, 2019; Simon-Campbell & Phelan, 2016). 


Learning Design Intervention 


Regarding the learning design of the interventions, it varies how well described it is. In the following, we 
identify four themes that emerge: 


e Duration. The duration of the interventions is one of the most commonly described parameters of the 
learning design. The duration varies greatly, ranging from a four hour session (Morente et al., 2014) to an 
experiment that stretches over two semesters (Malkemes & Phelan, 2017). The remaining studies are 
somewhere in between and of a length similar to a semester. The length of the intervention may greatly 
impact the outcome of the use, as longer use time causes familiarization with the adaptive technology, 
and the novelty effect will eventually decrease. Nursing students currently tend to prefer physical 
textbooks (Mennenga, 2016), and the introduction of electronic books embedded in a learning technology 
that dynamically changes according to the students’ performance may challenge some of the cultural 
assumptions and expectations of students. Thus, time to familiarize may be of importance when 
introducing ALT. 


e Placement of student interaction with technology in time. Only two of the studies describe when students 
were encouraged to engage with the adaptive technology. Hinkle (2020) describes an extensive use, 
where students were using the adaptive technology both before, after and during class. In Morente, 
technology was exclusively used in class (2014). Placement of student AL-interaction in time is crucial to 
the learning design, as it conditions the pedagogical aim of the use. Used before class, it may serve as 
preparation and provide the instructor with relevant data in which part of the course content to focus on 
and how to possibly remediate it. Used in class, it may serve as a substitute for teaching by a human 
instructor in certain parts of the curriculum. Used after class, it may serve as an evaluation tool to assess 
learning achievement. 
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e Engagement. Four of the studies described how they encouraged the students to engage with the ALT. 
The degree of formalization varies greatly, ranging from Hinkle and Moskal (2018), which had no 
requirements from the students, and their interactions and achievements were not taken into account to 
Malkemes and Phelan (2017), who had set criteria for engagement in terms of number of questions to be 
answered each week, a certain mastery level to be reached and students’ performance were taken into 
account in their final grade. In Hinkle (2018), the instructors continuously encouraged students to use the 
technology, and in Simon-Campbell and Phelan (2016), it is mentioned that the visualization of progress 
within the interface was a motivational factor for engagement for students. The extent of student 
engagement with the technology is evidently of importance to the outcome. In the learning design it is 
advisable to decide how properly to engage students, whether in terms of weekly activity, a target 
mastery level or counting their performance as part of final grade. 


e Agency. With the key feature of adapting to each student’s specific needs or level, the technology also 
holds a potential to take on some of the work that has previously been that of the teacher: To present 
content and scaffolding students in their learning process. In two of the studies, the agency and roles of 
the student, the instructor and the technology are addressed. In Morente et al., technology is used as a 
substitute for the instructor rendering her or him superfluous (2014). In Malkemes and Phelan, two 
different modes of agency are described (2017). In the first one, the instructor is assigned the role of a 
quiz designer, which means the instructor configures the technology by setting a target mastery level, 
deciding the number of questions to be answered and which parts of the course content the students 
should engage with. In the second mode, students are themselves given all the privileges described, and 
thus they obtain the role of conditioning their learning path themselves. None of the remaining six studies 
address this balance in their intervention. 


DISCUSSION 


In this section we discuss the findings and suggestions from the reviewed articles and draw on research from 
instructional design to expand on future directions of ALT in nursing education. In their meta-analysis of the 
efficacy of adaptive e-learning among health professionals, Fontaine et. al. concludes: “researchers should 
focus on research designs allowing the assessment of the impact of multiple educational design variations” 
(2019, p. 15). The studies included in this review echo this point, but also point to specific designs future 
studies may benefit from exploring. One study (Simon-Campbell & Phelan, 2016) draws attention to the 
possibility of informing classroom instruction with data from the students’ interactions with the technology, 
and thus focus on topics and questions, they have found particularly hard. A concern, in this regard, is how 
instructors properly remediate the content to improve learning in the classroom. Another concern is how to 
develop a study culture, where students do their ‘homework’ within the system timely enough for the 
instructor to take data into consideration before class. This requires that the instructor possesses a certain 
level of data literacy and is familiar with the technology used as well as its functionalities and interface. In 
other words it requires competency development of instructors. In the article, Simon-Campbell and Phelan 
(2016) suggest that policies are demanded to meet this and to set a certain level of engagement from 
students may be a way to ensure a minimum engagement. Literature on instructional design shows that 
instructors need to possess a certain technological pedagogical knowledge in order to realize the potential 
benefits from a given technology. This kind of knowledge is combines knowledge of pedagogy and teaching 
in a particular subject (e.g., science subjects within nursing education) with knowledge of how the technology 
works (Mishra, 2016). With ALT this is particularly demanding and important, as adaptive technology has a 
high degree of complexity in terms of algorithms and computation of data. An increased effort in sharing and 
developing such knowledge among instructors may positively impact some of the factors highlighted in the 
papers, such as motivation for use, engagement and the distribution of agency across technology and 
instructor. 


e@ From the perspective of adaptive learning, a unique feature of nursing education is its combination of 
declarative knowledge and cognitive skills on the one hand and procedural knowledge and clinical skills 
on the other. There is evidence, from especially medicine education, that adaptive technology is 
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particularly effective in learning cognitive skills and declarative knowledge, whereas the impact on 
procedural and clinical skills is less clear (Fontaine et al., 2019). This suggests that ALT may be particularly 
useful in core science subjects and should be integrated in a manner where it partly takes the agency and 
role of the teacher and the traditional textbooks. Kellman and Krasne (2018) present an approach to 
combine declarative and procedural knowledge in order to train novices to achieve expert level of 
recognition and fluency for diagnosis and category learning using adaptive technology. The key is for the 
students to encounter sufficient variation within and between categories in order to train accurate 
generalization and recognition of new exemplars. Adopting adaptive technologies for such purposes does 
not sideline the instructor as “it is the instructor’s responsibilities to assist students in understanding the 
functionalities and the value of the adaptive system” (Cavanagh et al., 2020, s. 186). Rather, the ALT 
should be viewed as an augmentation and modification of traditional practices (Romrell et al., 2014). As 
one of the articles point out, adaptive learning may also be useful in customizing patient cases related to 
learning of clinical skills (Hinkle & Moskal, 2018) and thus not substitute, but rather augment and modify 
learning practices. In fact, one of the key features of adaptive learning is that it may free more time for 
students and instructors to work on clinical skills and procedural knowledge while in the classroom. 
Drawing on those insights, we suggest that future practices of designing learning with adaptive 
technologies in nursing education takes point of departure in the following question: Formalia: e.g., What 
is the duration of the course involving adaptive learning? How many students and teachers are involved? 
Which parameters are mandated (duration of class, evaluation forms, etc.)? 


e What conceptions of learning undergirds the design? 
e What are the criteria for engagement, and how are these enforced? 
e What courses are deemed particularly well suited for adaptive learning and why? 


e What connection between what is learned (and what is not) with the ALT and the face-to-face or 
synchronous learning is hypothesized? 


In future studies of ALT, the learning design could greatly benefit from a larger degree of transparency and 
explication of the learning design involved, as well as arguments for what subjects adaptive technology are 
used for and why, and at what stage they are introduced in the course of education. Within the literature of 
blended learning and flipped classroom in nursing education, such inspiration may be identified (e.g., Cho & 
Kim, 2019; Li et al., 2019; Liang et al., 2017; McCutcheon et al., 2015). 


CONCLUSION 


In this scoping review, we set out to answer the question: What characterizes learning designs where 
adaptive learning technologies have been blended into nursing education? Through analysis of the six 
included papers, we have identified five themes, according to which each article is analyzed. The themes are: 
motivation, context, methods, technological intervention and learning design intervention. They vary along 
those five themes, and particularly the learning design is divided into further subthemes, that are worth 
attending to in instructional design in nursing education. Those are duration of intervention, engagement, 
placement of interaction in time and the distribution of agency between instructor and AL. 


However, it is also evident that the studies reviewed only sporadically describe the learning design of their 
interventions. We suggest future research and practice with ALT in nursing education addresses the learning 
design more explicitly. Experimenting with duration, placement of interaction and agency and roles may 
challenge or complicate conducting experimental research with pre- and post-test as well as intervention 
and control groups. However, we believe that more qualitative oriented design-experiments may pave the 
road for identifying new ways of designing nursing education, where the identified potentials of adaptive 
learning technology are fully realized. 
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